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DECLARATION UNDER RULE 1.131 



I, Sandra K> Darling, do hereby declare and say: 

1. I am the inventor in the above-referenced application. My home address is 743 West 
Nolan Way, Chandler, AZ, 85248. 

2. My invention in the above-referenced applicant was conceived and reduced to practice 
before the November 27, 2002, filing date of U.S. Patent 6,808,392 to Walton. 

3. I conceived my invention in 1998; after reviewing a draft report by Robert Marzano and 
published by the Mid-continent Regional Education Laboratory (McREL), I determined to use 
the expounded meta-analysis in a method to align instructional strategies with standards or 
benchmarks. I requested and received permission to reproduce Marzano 's material in connection 
with my process. Attachments 1 and 2 are letters from McREL granting permission, received in 
March and April 1999. 

4. Thereafter, I proceeded to develop a methodology to systematically prepare a database 
that could be used to align benchmarks and instructional strategies in accordance with Marzano' s 
knowledge domains and brain processing function categories. In early 2000, 1 assembled a team 
to undertake the task of classifying teaching practice and classifying learning benchmarks, then 



mapping teaching practices to the benchmarks. Attachment 3 is an exchange of email 
correspondence on February 17, 2000, with one of my team members. Attachment 4 is a March 
7, 2000, message from me to the team outlining the tasks in implementing my method. 

5. Attachment 5 is an alignment meeting agenda for performing the alignment tasks on 
April 6-9, 2000. The bulk of the alignment was done in the summer of 2000, but the process of 
refining, checking and making consistent the database continued into 2001 . 

6. In January 2001 I sent a draft patent specification to my attorneys. While admittedly 
rough, this draft included all the steps in what is now my claimed method. Attachment 6 is my 
cover email and the accompanying draft specification. 

7. In mid-2001 I convinced the Minneapolis Public School System to undertake a pilot 
study of the impact of applying my alignment method at certain low-performing elementary 
schools. The effect of implementation of my aligned instruction strategies was evaluated in 
detail by McREL over a two-year period. Attachment 7 is a copy of the study agreement with 
McREL. 

8. In connection with that study, I personally delivered an aligned instructional intervention 
plan to Bryn Mawr Elementary school on September 10, 2001, This was the first application of 
my method in a live environment Attachment 8 is a partial copy of the plan. 

I further declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and that these statements were 
made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code, and that 
such willful false statements may jeopardize the validity of the application or any patents issuing 
thereon. 
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ATTACHMENT 1 




Making a Difference 

in the quality of education and learning for all through 
excellence in applied research, product development, and service. 



Permission to Reprint 

March 18, 1999 

Permission is hereby granted to Learning Bridges to use in EDCI 542, "Instructional Strategies 
That Impact Academic Achievement" the following material, which was published by Mid- 
continent Regional Educational Laboratory: 

Effect sizes reported in A Theory-Based Meta-Analysis of Research on Instruction, by Robert 
Marzano. Aurora, CO: McREL, 1998. 

The material being used should be cited in the following manner: 

Marzano, Robert (1998). A Theory-Based Meta-Analysis of Research on Instruction. Aurora, 
CO: McREL. Reprinted with permission of McREL. 

: It should also be included in the bibliography, if any. 

; This permission is for noncommercial use only, expires on one year from the above date and may 
be renewed only by obtaining further written permission from McREL. This permission is also 
limited to the use and materials specified above. Any change in the use or materials from that 
specified above requires additional written permission from McREL before such use is made. • 

; ; This report can be purchased for $20 including shipping and handling from McREL, ATTN: 

" ;R?sp 

; Sincerely, \ 

LindaBrannan 
Senior Associate 

cc: D.Frost V 



Mid-continent Regional Educational Laboratory j 

2550 South Parker Road, Suite 500 • Aurora, CO 80014-1678 j 

303-337-0990 • fax: 303-337-3005 j 

www.mcrel.org • 



ATTACHMENT 2 



Making a Difference 

in the quality of education and learning for all through 
excellence in applied research, product development, and service. 

Learning Bridges 
743 West Nolan Way 
" Chandler, AZ 85248 

Permission to Use McREL Material 

April 13, 1999 

i:S ? Pennission is hereby granted to Learning Bridges to use in its workshops the following material 
| : wMph w^ pro by McREL: 

i ... Marzano, Robert ( 1 999). A Theory-Based Meta-Analysis of Research on Instruction, 
i ; The material should be cited in the following manner: 

/ ^Marzano, Robert (1999). A Theorv-Based Meta-Anal ysis of Research on Instruction, Aurora, 
( L CO: McREL Online: www.mcrel.org/standards/Usedby permission of McREL. Online: 
www.mcrel.org/standards/articles/meta.asp 

It should also be included in the bibliography, if any. 

: ;■; This permission relates to the use of this material in Learning Bridges workshops, including: 

f • Brain, Behavior & Student Responsibility 

- • Constructing Meaning and Making Connections: Human Brain & Learning 
I f V /'Dimensions of Learning, Brain Research & Instruction 

^ ^ v ! Teaphing the Way the Learns 

^ 1 ^^' ; ^!^® C Teaching and Learning ■ : 

y This permission expires on one year from the above date and may be renewed only by obtaining U 
further written permission from McREL. This permission is also limited to the use and materials 

- specified above. Any change in the use or materials from that specified above requires additional 
written permission from McREL before such use is made. 



'Sincerely, 




Vice President, New Business Planning and 
New Market Development 

Mid-continent Regional Educational Laboratory 
2550 South Parker Road, Suite 500 • Aurora, CO 80014-1678 
303-337-0990 • fax: 303-337-3005 
www.mcrel.org 
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Dr. Sandra Darling ^ ;/f) ^ riML .^ 

From: "Michelle Beach" <beachmi@mhd1.moorhead.msus.edu> CUtf t|tVK *^fi^fi-AJjcA, 

To: "Dr. Sandra Darling" <sdarling@learningbridges.com> «J~ <& T-rr-c "i f TT" _Q 

Sent: Thursday, February 17, 2000 8:32 PM r> / • I .. A ~**"r» . , 

Subject: Re: The Incredible Dream Team Kit" I tro S/Q * 

Hi Kiddo! 

My preference is choice B, Thursday March 30 - Sunday April 2. I teach classes all day 
Mondays, so choice A wouldn't work so well. 

Now you ask if that is my final choice, and announce that my answer is correct and I win a 
million dollars! 
: ) Michelle 

"Dr. Sandra Darling" wrote: 

The Incredible Dream Team consists of these 9 members: Kris Nei - 
knei@leaminqbridges.comMarcia Schuldt - mschuldtOle arninabridges cnmMir.hPiiP Beach - 
beachmi@mhd1.moorhead.msus.edu Donna Tileston - dwtileston@vahoo.com Nancy Oelklaus - 
noeJjdaus@aoLcomRhia Roberts - rhial KSnetzero.net Gwen Doty - boss302(3>northlink.com Terri 
Mesna - tmesna@palomar.eduSandy Darling - sdarlinQ@learninabridq es.com WE ARE GOING TO 
BUILD THE DATABASE THAT GENERATES THE INSTRUCTIONAL INTERVENTION PLAN. 
That's our mission. Here's how we will do it. Task 1: Determining what is an instructional 
strategy. Who: Kris will work with others to provide a clear definition of "Instructional Strategy" that 
will guide our search. That team will use concept attainment to help them with definition. Task 2: 
Identifying the instructional strategies that need to go into the database. Who: THE ENTIRE 
TEAM How: I will send you for starters, bibliographies from Marzano's Meta-Analysis, from Models 
of Teaching, and anything else I can get my hands on. YOU have many, many more. Send them 
We will split up the work. For instance, I will divide up the 30 - 40 pages of research studies from 
Models of Teaching, and send SOME to each of you (minus Tern and Rhia - who have different 
jobs). We researchers will pull the research from ERIC, the Net, and anyplace else we can find 
some. Once identified, we will pull specific data (which I will list for you) and send it via e-mail to 
Rhia Roberts. Our goal as researchers is to have identified over a thousand instructional strategies, 
grounded in research, to put into our database. WHAT DOES RHIA NEED FROM EACH OF THE 

RESEARCH STUDIES ON INSTRUCTION? 1. Name of the Instructional Strategy2 Short 1 

Description of the Instructional Strategy (2 - 3 sentences)3. Long Description of the Instructional l 
Strategy (1 paragraph)4. Citation for the Research (you know... author, title, etc.)5. Mean of the V 
expenmental group on whatever measure they're using6. Mean of the control grou p of whatever Q 
measure they're using7. Standard deviation of the control group. (Rhia said they usually report this O / /) 
but if they report individual scores, then send those and she will compute the standard deviation^ ' ^ y ^-d- 
Number of People in the student - n = number9. Length of the experiment - timewiselO Training of 
the expenmenters. Whoever is participating in doing the instructional strategy, were they given 
training, and if so, what was it.1 1 . The reliability and validity of the tests used in the study. For 
example, if they used a test to measure a change in reading or math, what is the reliability and 
validity of that test. If they don't provide that, be sure to put the test names in. Rhia has the 

reliability and validity data on a number of tests and can look it up. 12. The gender of the ) 

participants, if available. So there are 12 things that we need to pull from each study. We send 
these 12 things for each study to Rhia Roberts through e-mail. Task 3: Computation of Effect Size 
and Percentile Gain Scores for every instructional strategy submitted. Rhia will use Glass' 
methodology to compute these two scores. Who: Rhia Roberts Task 4: Putting all of this data into 
an Access Database for safe-keeping. Who: Terri Mesna - Terri will put all of the researchers' 12 
things plus 2 more pieces of data from Rhia into an Access database. Each instructional strategy 
will have a number for identification. Terri will also enter the McREL Mathematics and Language 
Arts Standards into the Access Database. Task 5: Mapping the Instructional Strategies to the 
McREL Standards in Language Arts and Mathematics. The Incredible Dream Team will get together 
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the last week of March (let me know if your preference) Choice A: Sunday, March 26 - 
Wednesday, March 29 Choice B: Thursday, March 30 - Sunday, April 2 We will take 2 days 
to achieve consensus on which strategies are mapped to which McREL Language Arts 
standards. We will take 2 days to achieve consensus on which strategies are mapped to which 
McREL Mathematics standards. Terri will put the identification numbers of the mapped strategies in 
a separate field by standard. Task 6 Celebrate!! We will have all of the raw data ready for the 
technician to build our custom database to generate the Instructional Intervention Plan. Do 
you think it is possible for the 7 of us to EACH come up with 215 strategies supported by 
research by the third week of March? Can we tap enough resources to do that? Shall we 
give it a try? YOU BETCHA....a MNism. I will get the bibliographies copied and out in the mail 
TOMORROW. Please do not think it's an inclusive list. You all have many, many more. I know 
you'll all feel like youYe back in school working on your master's or doctorate all over again. I 
promise you. ..if we can get this done together, we shall share in the rewards together, too. Print out 
and save this sheet so you have a list of everyone's e-mail addresses. Periodically, I'll have Terri let 
us know how far we are.. .how many are in the database. She'll have the number. Lefs go for 
it! Sandra K. Darling, Ph.D. 
Learning Bridges 
743 W. Nolan Way 
Chandler, AZ 85248 
480-883-0704 phone 
480-883-0705 lax 

sdarlinq@learningbridqes.com e-mail 
Web site www.leaminqbridges.com 
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ATTACHMENT 4 



March 7, 2000 



TO: Kris Nei 

Marcia Schuldt 
Michelle Beach 
Donna Tileston 
Nancy Oelklaus 
Rhia Roberts 
Gwen Doty 
Bruce Laird 
Terri Mesna 

FROM: Sandy Darling 

RE: Instructional Database 

I've heard from almost all of you regarding the April 6-9, 2000 dates. Hopefully, 
by the end of today, I'll have heard from all of you. The purpose of this note to all 
of you is four-fold: 

1 . Outline the tasks with a strategy for accomplishing them. 

2. Define a timeline for completion 

3. Propose a compensation plan with its risks and contingencies 

4. Get a firm commitment from you on participation. 



I've learned a couple of things, and Nancy Oelklaus was really helpful in defining 
this job, too. The tasks before us, as I see it now, are these: 



TASK and STRATEGY 


TIMELINE 


TASK ONE Identify all of the "Teaching Practices" 
or "Professional Practices" that we can from a) 
our research, b) our own experiences, and c)from our 
own resources. I will send you the list that I have 
from Marzano's meta-analysis with their 
accompanying effect sizes for you to examine. 

Teaching practices are those behaviors that 
teachers do which are alterable (in other words. 


By March 31 , 2000 we 
will have as many 
teaching practices as 
possible with supporting 
research in an Access 
database with at least 
one supporting citation 
as to the research base 
which supports it. Some 
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Learning Bridges could provide instruction on them 
and the teaching practices would change). 

If your identified teaching practices have been 
supported by research which indicates its 
effectiveness or impact on learning, please see if you 
can find the data in order for Rhia Roberts to compute 
the effect size on it. You already have that 
information from me. Many, many new teaching 
practices (particularly those which were identified 
from research on the brain, multiple intelligences, 
etc.) have not had effect size studies done on them. 
We will include them in our "list" anyway. Those 
teaching practices (without effect size studies) are 
supported by other types of research and best 
practice studies. 

Some teaching practices address variables that 
impact on academic achievement in a general way, 
but are not specific to language arts or mathematics. 
INCLUDE THEM. We will put thorn into a category 
for Other Practices Which Impact Achievement (or a 
better title if we can think of one). These might 
include tutoring, classroom climate issues, classroom 
management issues, etc. 

TO DO: E-MAIL YOUR LISTS OF TEACHING 
PRACTICES TO TERRI MESNA AT 

tmesna(5)palomar.edu 

IF YOUR TEACHING PRACTICE HAS AN 
IDENTIFIED "EFFECT SIZE," INCLUDE THAT AS 
WELL IN YOUR E-MAIL TO TERRI. 


of the teaching practices 
will have an Effect Size 
associated with them as 
well; some will not. 

By March 10, 2000 you 
should have a copy of 
Sandy's notes on the 
meta-analysis with their 
accompanying Effect 
Sizes by Priority Mail. 


TASK TWO Identify resources where there is 
information on these teaching practices, i.e., books, 
articles, research studies, web sites, videos, etc. 

Each teaching practice will have many, many 
references which support it. You may personally not 
1 ia vc a» u mm. wmer team memDers can contribute 
resources, books, articles, research studies, web 
sites, etc. to the teaching practice that you submitted. 

We will all use APA format as the acceptable means 


March 31, 2000 -A list 
of bibliographic 
resources will exist in an 
Access database. Each 
of the resources will 
have a unique, 
identifying number 
assigned by Terri (as 
she creates that table in 
the database from the 
material that you 



2 



of submitting bibliographic data so that there is 
uniformity in what the customer sees. 

Examine the references which were sent to you (the 
big pack!) for possible sources which support the 
teaching practices you submit. Include them where 
you can. 

TO DO: E-MAIL YOUR BIBLIOGRAPHIC DATA TO 
TERRI MESNA AT 

tmesna@Dalomar.edu 

IF THE RESEARCH HAS DATA FROM WHICH AN 
EFFECT SIZE MAY BE COMPUTED, E-MAIL THAT 
DATA TO RHIA ROBERTS AT 

rhia11(3>netzero.net 


submit.) That identifying 
number will be 
"attached" or "assigned" 
to all of the teaching 
practices which the 
bibliographic source 
supports. 


TASK THREE Review the McREL Standards and 
Benchmarks for grade level clusters of: a) K-2, 3- 
5, 6 - 8, and 9 - 12 in Language Arts and 
Mathematics. These are the content standards 
against which we will align. Sandy will be responsible 
for getting each of you a copy of them. 

I u uu. KNUW AND UNDERSTAND WHAT THE 
CONTENT STANDARDS (AT THE GRADE LEVEL 
CLUSTER LEVEL) MEAN PRIOR TO THE 
MAPPING MEETING. 


Friday, March 10, 2000 
you should have the 
McREL Language Arts 
and Mathematics 
Content Standards. 
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TASK FOUR Writing long and short descriptions of 
the teaching practices as they would apply to the 
mathematics and language arts content standards 
(from the McREL grade level clusters). 

Nancy Oelklaus has graciously volunteered to assist 
with this task. If you need help, please contact her at 

noelklaus(2>aol.com 


March 31, 2000 the long 
and short descriptions of 
the teaching practices 
will be entered into an 
Access database. 

Please remember to get 
them to Terri with 
enough time for her to do 
the data entry coming 
from all of you. We want 
HER job done prior to 
the mapping meeting. 


or 1-800-717-2723 X14. The phone number is for 
work, so use discretion please. 

The descriptions of the teaching practices will be 
different for a K - 2 math standard than a 9 - 12 
standard, and perhaps different for language arts and 
mathematics. Focus on the application of the specific 
learning being asked for in the benchmark standard. 

We will be "mapping" teaching practices to the 
benchmark standards by the grade level clusters, and 
not just the standard statement itself. The resulting 
information will be more useful to classroom teachers. 
(Thanks to Nancy for this valuable suggestion!) 

TO DO: E-MAIL YOUR LONG AND SHORT 
DESCRIPTIONS OF THE TEACHING PRACTICES 
TO TERRI MESNA AT 

tmesna(3>Dalomar.edu 


THEY MAY BE MORE GENERIC DESCRIPTIONS 
AT THIS POINT IN TIME, SINCE WE WILL NOT 
HAVE MAPPED THEM TO CONTENT YET. WE 
WILL RE-WRITE THEM FOR THE SPECIFIC 
BENCHMARK STANDARDS FOLLOWING THE 
MAPPING PROCESS. 


TASK FOUR - Mapping of the teaching practices to 
the benchmark standards of McREL in both language 
arts and mathematics. 

By this time, we will have a number of things in our 
database for safe-keeping: 

a) McREL's standards and benchmarks (by grade 


April 6-9, 2000 
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level clusters) Terri will have assigned them a 
unique, identifying number. 

b) Teaching practices (Terri will have assigned 
them a unique, identifying number.) 

c) Descriptions (long and short) of teaching 
practices 

d) Effect Sizes for SOME of the teaching 
practices 

e) Percentile Gains for SOME of the teaching 
practices 

f) Bibliographic Data which has been assigned to 
teaching practices 

For TWO days, we will map the list of teaching 
practices to the 223 mathematics benchmarks. For 
TWO days, we will map the list of teaching practices 
to the 275 language arts benchmarks. 

We will accomplish this, as a team, based upon our 
professional judgment utilizing all of the 
knowledge and experience that we bring to the 
table. Worksheets will be provided for our use. 

Bring any resources to the task that you feel may be 
useful. 

TO DO: AT THE LEARNING BRIDGES OFFICES, 
THE TEAM WILL MEET FOR 4 DAYS TO "ALIGN" 
INSTRUCTION TO STANDARDS. 






TASK SIX The results of the mapping will be entered 
into the Access database for safe-keeping. Terri will 
do this from our worksheets. 

TO DO: TERRI ONLY Data Entry 


Friday, April 21, 2000 

The unique numbers of 
the teaching practices 
are assigned to the 
benchmark standards. 


TASK SEVEN Rewrite the teaching practices 
descriptions to best fit the grade level clusters of 
McREL benchmark standards for language arts and 
mathematics. These rewritten teaching practices will 
be entered into the database. 

TO DO: REWRITE THE TEACHING PRACTICES 
DESCRIPTIONS. E-MAIL THE REWRITTEN 
DESCRIPTIONS TO TERRI AT 


May 5, 2000 

Final teaching practices 
descriptions are entered 
into the Access 
database. 
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tmesna@palomar.edu 




FOR ENTRY INTO THE DATABASE. 


TASK EIGHT Rank ordering the teaching practices 
in order of their impact on learning based on a) the 
professional judgment of the Team, and b) the Effect 
Size when available. The rank-ordering of the 
teaching practices will be under the McREL 
benchmark clusters. A pair-weighting process will be 
utilized by each member individually. The paired 
weighting forms and directions will be mailed to each 
member of the team to be completed. Responses will 
need to be returned to Learning Bridges by May 15, 
2000. The composite created by Sandy will be the 
team's final rank order. 

TO DO: COMPLETE THE PAIRED WEIGHTING 
FORMS TO RANK ORDER THE TEACHING 
PRACTICES A) UNDER EACH OF THE 223 MATH 
BENCHMARKS AND B) UNDER EACH OF THE 
275 LANGUAGE ARTS BENCHMARKS. MAIL THE 
COMPLETED FORMS TO LEARNING BRIDGES IN 
THE ENVELOPES PROVIDED. 

CREATE THE COMPOSITE: SANDY ONLY 
ENTER THE RANKING INTO THE DATABASE FOR 
EACH OF THE BENCHMARKS - TERRI ONLY 


May 19, 2000 

The teaching practices 
are assigned a ranking 
number within each 
benchmark standard. 


ALL DATA WILL BE READY FOR THE DATABASE 
BUILD ITSELF 


May 19, 2000 



COMPENSATION AND RISKS 

Learning Bridges is seeking funding from a variety of sources, including business 
partnerships (Classroom Connect, Intel, Apex, Wrrr), venture capital 
(EduVentures, Bluefire, St. Paul Companies, Vulcan Ventures), AZ Small 
Business Association loan, offering a specialized deal to Nebraska, Crowne Point 
Technologies and X-Nova (Chuck Lowry and Bob Weathers) are looking at 
possible funding for us, and even some of you have contacted organizations and 
individuals to explore funding options. Classroom Connect and Apex have 
already indicated a willingness to work with us to take our product to market IF 
we have the database built. The U.S. Department of Education (OERI) has 
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indicated it will pay for the research evaluation project part of the proposal 
because they WANT the database to exist. 

As Nancy said, we're stirring the pot a lot; it WILL make good soup. However, 
right now Learning Bridges does not have the money to pay for your time. What I 
am proposing is this: When, AND IF, we get the funding for our project, I will pay 
each of you $15,000 for your contribution to this project. It will be a "work for 
hire" sort of deal. You will not own the database; Learning Bridges will. The risk 
for you is that we may not get funding to do the database build itself, which is a 
minimum of $150,000 to $200,000. 

Basically, if we get funded, you get paid. If we do not get funded, you will not get 
paid. That is the risk you need to evaluate for yourself. 

I know that, personally, I believe right to the bottom of my toes that we will find 
funding. I have leveraged every asset that I have, except my sons (but they've 
had to help, too). I want this venture to be successful because it will make a 
difference in teaching and learning, and I care about that very deeply as do you. 
I also want this venture to be successful because I want Learning Bridges to 
make money... lots... why not? It has taken tenacity, courage, tears, tolerance for 
ambiguity and uncertainty, and sometimes, laughing out loud, to get this far... not 
just for me, but for every one of you who has been a part of this Learning Bridges 
journey to "Become a Bridge to a New Way of Teaching and Learning." We have 
been supported and encouraged by educators and business people all over the 
country, some of whom have given dearly of their time and expertise in their 
belief in Learning Bridges. My biggest fear is to quit one month too early! 

When we get funded, our first endeavor (besides beginning the database build 
which will generate the Instructional Intervention Plan) will be to engage in 
Strategic Planning. The entire Learning Bridges Team will be invited to 
participate in establishing a five year Strategic Plan to "Plant our Flag into the 
Future." We will complete an action plan that identifies the roads we will take to 
achieve our vision. There are many options for being a part of Learning Bridges 
if we map our journey well. We can create options for you to share in the 
benefits as well. 

But, first things first (as Covey would say), we need to get the data to build the 
database. I need a commitment from you to take the risk, and to share your gifts 
to make it happen. I don't want to assume. After you've looked over those big 8 
tasks to be done, please let me know this week if you are willing to be a part of 
this task... and what a task it is!! 

I am grateful and humbled by your support for me and for Learning Bridges. You 
can count on me to do my part to the very best of my ability. 

Thank you for being you! 
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LEARNING BRIDGES™ 
INSTRUCTIONAL DATABASE ALIGNMENT 
Confidential Property of Learning Bridges™ 
April 6-9, 2000 

AGENDA 

Thursday, April 6 

8:00 Continental Breakfast 

8:30 Review of Process 

Confidentiality Statements 
9:00 Combining/Synthesizing Teaching 

Practices 

10:00 Alignment to 223 Math 

Benchmark Standards 

12:00 Lunch 

1 ; 00 Continue Math Alignment 

5:00 Happy Hour with Margaritas 

3 P's 

Lasagna Dinner 

For the 3 P's - bring in 3 items 
which represent something 
Personal, something Professional 
and something that represents your 
Passion. We'll get to know each 
other better with the sharing, 
and it'll be fun! 




Friday, April 7 

8 : 00 Continental Breakfast 

8 : 30 Continue Math Alignment 

12:00 Lunch 

* : 00 Continue Math Alignment 

5:00 Adjourn - Dinner on your own 

Saturday, April 8 

8:00 Continental Breakfast 

8:30 Alignment to 275 Language 

Arts Benchmarks 

12:00 Lunch 

1 ; 00 Continue Language Arts 

5:00 Adjourn - Dinner on your own 

Sunday, April 9 

9:30 Continental Breakfast 

10:00 Continue Language Arts 

12:00 Lunch 

1 : 00 Continue Language Arts 

4:00 Next Steps 

5:00 Adjourn 



I know you all have varying times you have to leave. We'll try 
to coordinate with you. I'm really looking forward to seeing 
all of you, and beginning the first steps of creating an Aligned 
Instructional Database. The weather is SUPPOSED to be in 
the 80s so bring shorts! Sandy 

Arrival Times 

Michelle - Wednesday, April 5 th - 1 :29 PM 
Terri - Friday, April 7 tfi - 9:00 PM 
Marcia - Thursday, April 6 th - 8:32 PM 
Gwen - Thursday, April 6 th - 12:00 PM 
Bruce - Wednesday, April 5 th - 7:24 PM 
Kris - Tuesday, April 4 th - 5:55 PM 
Donna - Wednesday, April 5 th - 2:45 PM 
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Mott, Joseph W. 

From: Dr. Sandra K. Darling [sdarling@learningbridges.com] 

Sent: Tuesday, January 09, 2001 1 :1 9 PM 

To: Mott, Joseph W. 

Cc: Chuck Lowry 

Subject: Patent 

Attachments: Patent Application.doc; Non-Provisional Application for a Patent.doc 
Dr. Mr. Mott, 

Attached are 2 documents to start the patent application process. I have a couple of questions, and you probably 
will have many after looking at them. 

a) Patent Application (I tried to make it like the form required. Are you the "Applicant" or is Learning Bridges?) 

b) Non-Provision Application for a Patent.. .I'm not sure this is what you need, but with your guidance, I'll get it 

Call me. 480-883-0704 Thanks! S 
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INVENTOR INFORMATION 

Inventor One Given Name:: 
Family Name:: 



Sandra Kay 
Darling 



Postal Address Line One- 
City:: 

State or Province:: 
Postal or Zip Code: 



743 W Nolan Way 

Chandler 

AZ 

85248 



Citizenship Country- 



USA 



Given Name of Applicant:: Joseph W 

Family Name: Mott 

Authority Under : : 1 1 
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Non-Provisional Application for a Patent 

(1) A written document which comprises a specification (description and 
claims), and an oath or declaration; 

(2) A drawing in those cases in which a drawing is necessary; and 

(3) A filing fee. 

Process or Concept Patent Idea 
Title: 

Alignment of Instructional or Training Strategies with Specific Kinds of 
Knowledge (e.g., Declarative and Procedural) Based on How the Brain 
Processes Specific Kinds of Knowledge in Rank Order Based on Effect Size on 
Learning. 

Theoretical Underpinnings 

Knowledge 

The knowledge domains include information, mental processes, and 
psychomotor skills. Learning expectations that address knowledge (e.g., State 
Content Standards for K - 12, McREL Compendium of Standards) encompass 
two types of knowledge - declarative and procedural - which are defined by 
Robert J. Marzano in "A Theory Based Meta-analysis of Research on Instruction" 
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(1998). The brain's linguistic processor "encodes experience into memory as 
propositions organized into declarative and procedural networks. These 
networks encode the 'what' of human knowledge and the 'how to,' both mental 
and physical" (p. 45). Marzano further defines declarative and procedural 
knowledge in Designing Standards-Based District. Schools, and Classrooms 
(Marzano and Kendall, 1996). "Declarative knowledge can be thought of as 
'information' and usually involves component parts.... Procedural knowledge 
consists of skills, strategies, and processes." The brain processes these two 
types of knowledge differently. This has implications for choosing instructional 
strategies to teach both kinds of knowledge based on how the brain processes 
each type of knowledge. 

Brain Processing 

The three systems of brain processing are the cognitive (which includes storage 
and retrieval processes, basic information processing, communication and 
knowledge utilization), metacognitive (which includes goal specification, process 
specification, process monitoring, and disposition monitoring), and the self 
system (which includes self-attributes, self and others, nature of the world, 
efficacy, and purpose - along with a mechanism that notes the difference 
between the desire status and the perceived status (Marzano, 1998, p. 107). 
Marzano reported three implications of brain processing of specific kinds of 
knowledge on classroom instruction. "Implication #1 : Teachers should identify 
the knowledge and skills that are the targets of instruction.... Implication #2: 
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Teachers should identify and use specific instructional techniques for specific 
instructional goals.. ..Implication #3: Teachers should regularly utilize 
instructional techniques that apply to all types of instructional goals" (pp. 201 - 
208). 

Effect Size 

The difference between an experimental and control group on the dependent 
measure into a standardized metric is referred to as an "effect size." (Glass, 
1976, 1978; Glass & Smith, 1979; Glass, McGraw & Smith, 1981) This is used 
when conducting a meta-analysis. The advantage is that effect sizes can be 
compared across students that use different dependent measures. Marzano 
(1998) states that the effect size metric is useful because "it is standard deviation 
units and can, therefore, be interpreted as a change in the percentile ranking of 
the 'average' subject in the experimental group." 

The instructional strategies chosen to be aligned to content standards 
(knowledge) had an Effect Size based on meta-analyses research representing 
30,000+ studies. The instructional strategies were ranked for effectiveness using 
a Paired Weighting Process based on Effect Size and the professional judgment 
of content and instructional specialists. 
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The Process 

State Content Standards for language arts and mathematics were examined to 
determine the type of knowledge they were expecting of students, i.e., 
declarative, procedural, as well as the type of brain processing required to 
demonstrate learning, i.e., storage and retrieval, information processing, etc. 
Thousands of teaching practices (instructional strategies) were identified that 
addressed the identified type of brain processing for a specific type of 
knowledge. The Effect Size of the instructional strategies was determined from 
the meta-analyses of research on instruction to identify the impact on learning in 
language arts and mathematics. 

The ranges identified for the Impact on Learning were: 

.01 - .40 = Low which at most takes learning to the 66 th percentile which is below 
mastery. 

.40 - .80 = Moderate which at most takes learning to the 79 th percentile which is 
right below the mastery level. 

.81 - 1 .3 = High which at most takes learning to the 90 th percentile - significantly 
above mastery. 

1 .31 + = superior which will take learning from the 91 st percentile to the 99 th 
percentile - usually considered superior in grading, too. 
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The teaching practices were mapped to the content standards in language arts 
and mathematics based on the type of brain processing required to demonstrate 
learning of a specific type of knowledge. 

Example of Mapping 



What is the 
standard? 



Language Arts 
Standard (or 
category) 6 - 

Demonstrates 
competence in the 
general skills and 
strategies for 
reading a variety of 
literary texts. 

Grades 3-5 
(Level II) 

Knows the defining 
characteristics of a 
variety of literary 
forms and genres 
(e.g., fairytales, 
folktales, fiction, 
nonfiction, myths, 
poems, fables, 
fantasies, historical 
fiction, biographies, 
autobiographies.) 



What kind of 
knowledge is 
represented by 
the standard? 



Declarative 
Knowledge 

"Declarative 
knowledge can be 
thought of as 
"information" and 
usually involves 
component parts. 
For example, 
knowledge of the 
concept of 
'democracy' 
includes an 
understanding 
that decisions are 
made by the 
people, that each 
person has a 
single vote, that 
votes are 
weighted equally, 
and so on. They 
involve an 
understanding of 
terms and 
concepts." 
(Marzano, 1996) 



How does the 
brain process that 
kind of 
knowledge? 



Information 
Processing 
Function of the 
Cognitive 
System- 
information being 
declarative 
knowledge about 
concepts, 
principles, ideas, 
or generalizations 



What strategy 
addresses that 
kind of brain 
processing for 
that kind of 

knowledge? 

Concept 
Attainment is an 
information 
processing 
teaching practice 
for discovering the 
identifying 
characteristics 
and discriminating 
attributes of a 
concept, principle, 
generalization or 
idea, and for 
reflecting on how 
the learning 
occurred. 
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The number of teaching practices for each grade level (benchmark) standard 
was reduced to 25 and NO teaching practices were included that did not have at 
least a moderate Impact on Learning. The 25 teaching practices or instructional 
strategies were then rank ordered by their Impact on Learning using both the 
Effect Size and professional judgment of content and instructional specialists. 

The result of the Process is a database of the top 25 teaching practices for the 
benchmark (grade level) standards of 49 states, ranked by the Impact on 
Learning (Effect Size), that specifically meet the type of brain processing required 
to learn the type of knowledge stated in the benchmark standard. 

Claim 

The alignment of instructional teaching practices to content standards and 
benchmarks has never been done. A review of the research underway and a 
review of the literature currently published was conducted by Sandra K. Darling 
and David Frost (Vice President for New Business Development, Mid-continent 
Research on Education and Learning - McREL). 

The alignment of teaching practices to state content standards and benchmarks 
based on brain processing (for a specific type of knowledge) has never been 
done. 
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The identification of the top teaching practices that are ranked for effectiveness 
(Effect Size) in learning state content standards and benchmarks has never been 
done. 

Further Research 

Researchers at the Mid-continent Research on Education and Learning (McREL) 
and Learning Bridges have applied for a Field Initiated Study (FIS) from the 
Office of Educational Research Institute (OERI) to measure the effectiveness of 
two (2) products originating from this concept and process: Learning Bridges 
Instructional Intervention Plan and Learning Bridges Web-Enabled Training on 
Instruction. Both are aligned to the specific content standards and benchmarks 
of 49 states. 

Respectfully Submitted: 
Sandra K. Darling, Ph.D. 
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ATTACHMENT 7 



AGREEMENT 



This Agreement is made this 28 day of August, 2001, by and between Learning Bridges 
("Company") and the Mid-continent Laboratory, Inc. identified above ("Contractor"). 



ARTICLE I 
Purpose and Scope of Work 

1.1 The purpose of this Agreement is to engage Contractor to perform the services 
described in the Scope of Work attached hereto as Schedule A and made a part hereof. 

1.2 Contractor shall at all times act in good faith and use its best efforts to perform all of 
the work provided for in this Agreement. 



ARTICLE II 
Payments to CONTRACTOR 

2.1 Company shall pay for all services provided and associated expenses incurred by 
Contractor under this Agreement on a fixed price basis the total sum of one hundred sixty one 
thousand seven hundred eight dollars ($161,708.00). 

2.2 Payments to Contractor shall be made at the times, in the manner and upon the 
conditions set forth in the Payment Schedule attached hereto as Schedule B and made a part hereof. 



ARTICLE III 
Term and Termination 

. 3.1 The period of this Agreement shall be from August 15, 2001 through September 30, 

2003. 

3.2 This Agreement may be terminated in whole or in part, at any time and for any reason, 
by either party at its discretion or for its convenience, upon at least 30 days prior written notice to the 
other party. In the event of such termination, Contractor shall be compensated under this Agreement 
through the effective date of termination, provided a final invoice is received by Company no later 
than 60 days after such date. 
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3.3 In the event of any breach of this Agreement by either party, or in the event either 
party is unable to perform any substantial part of this Agreement, this Agreement may be terminated 
immediately by either party by written notice to the other. 

3.4 If this Agreement is terminated by reason of a breach by either party, the other party 
shall be entitled to all available legal, equitable and administrative remedies for such breach, 
including without limitation, the right to all damages resulting from such breach and the right to an 
injunction restraining or compelling action in accordance with this Agreement. 



ARTICLE IV 
Reports and Audit 

4.1 Company and any of their duly authorized representatives, shall have full and timely 
access to all books, documents, papers, records and accounts of Contractor that are pertinent to this 
Agreement for the purpose of conducting such audits and examinations, or of making such excerpts 
or copies, as the examining party deems appropriate. Each party shall bear all of its own expenses in 
connection with any such audit, examination or copying, unless any such audit or examination reveals 
any knowing breach of this Agreement by Contractor or negligence of Contractor, in which event 
Contractor shall bear all costs of such audit or examination in addition to any other damages or 
liabilities resulting from such breach or negligence. 



ARTICLE V 

Ownership, Intellectual Property, and Pre-existing Works 

5.0 Subject to Section 5.2, Company shall own the materials provided by McREL in 
connection with the Scope of Work shown on Schedule A. However, Contractor shall retain all right, 
title and interest to its intellectual property embodied in or by such materials, and to its pre-existing 
works, consultations, trainings, analyses, theories, models, publications, research, evaluations, or 
assessments. 

5.2 Notwithstanding Section 5.1, if in connection with this Agreement, Contractor 
modifies or improves any materials that were created, published or copyrighted by Contractor prior to 
the effective date of this Agreement, and if such modifications or improvements are not so substantial 
as to make the modified or improved material a derivative work, then Contractor shall retain its 
ownership and copyright with respect to such material. 
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ARTICLE VI 



General Provisions 

6. 1 Contractor represents and warrants as follows: 

(a) Contractor is incorporated in and in good standing with the state of Colorado and is 
duly authorized to conduct business and is in good standing in each other state in which it shall 
conduct any activities under this Agreement. 

(b) Contractor has full power and authority to enter into this Agreement and carry out 
all of its terms and provisions. 

(c) This Agreement has been duly authorized by all necessary institutional action and 
represents the legally binding and fully enforceable obligation of the Contractor, and the person 
entering into and executing this Agreement on behalf of Contractor are fully authorized to do so. 

(d) No part of this Agreement or of any activity to be undertaken by Contractor 
hereunder shall violate or otherwise be inconsistent with any provision of 

(i) Contractor's articles of incorporation or bylaws, 

(ii) any contract, covenant, representation, warranty, license or indenture made by or otherwise 
applicable to Contractor, 

(iii) any judicial or administrative order or ruling applicable to Contractor, or 

(iv) any statute, rule or regulation applicable to Contractor or to Contractor's activities under this 
Agreement. 

6.2 In connection with its performance of this Agreement, Contractor shall comply with 
all applicable Federal and state laws, regulations, standards, orders and requirements. 

6.3 All notices, reports and other written communications to either party under this 
Agreement shall be hand delivered or shall be mailed, postage prepaid, by first-class, registered or 
certified mail, or sent by private courier or express company, to the address noted for each party at the 
beginning of this Agreement, or shall be transmitted by fax to the number noted for each party. 
Delivery shall be deemed to have occurred on the earlier of actual receipt by the recipient or, in the 
case of mailing, three days after deposit with the United States Postal Service. Either party may 
change its address or fax number for purposes of this Agreement at any time by written notice to the 
other party. 

6.4 In performing its services hereunder, Contractor is an independent contractor. Nothing 
herein shall create any partnership or joint venture between the parties or be construed as establishing 
any employment relationship. Contractor shall have no authority to speak for, act on behalf of or in 
any way bind or obligate Company, except as may be otherwise expressly provided herein. 

6.5 This Agreement may not be amended, modified or supplemented except by a written 
instrument signed by the party sought to be bound thereby. 
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6.6 Contractor may not assign all or any portion of this Agreement, or any of its rights, 
powers, duties or obligations hereunder, without the express written consent of Company. 



6.7 This document, together with all schedules, contains the entire Agreement of the 
parties with respect to the subject matter hereof and supersedes all prior communications, 
representations and agreements. 

6.8 Neither party shall limit or exclude any person from participation in any activity under 
this Agreement on the basis of race, color, creed, national origin, sex or disability. 

6.9 This Agreement is made in Aurora, Colorado and shall be governed by the laws of the 
State of Colorado. The parties consent to venue in the courts of either Arapahoe County, Colorado or 
Denver County, Colorado with respect to any dispute arising hereunder. 

6. 1 0 Points of contact shall be as follows: 



IN WITNESS WHEREOF the parties have executed this Agreement on the dates set forth 
below to be effective as of the date first above written. 



Dr. Sandra K. Darling 
Learning Bridges 



Dr. Elisabeth Palmer 
Mid-continent Laboratory, Inc. 
2550 S. Parker Road, Suite 500 
Aurora, CO 80014-1678 
Telephone: 303-632-5559 
Fax: 303-337-3005 



743 W. Nolan Way 
Chandler, AZ 85248 



Telephone: 480-883-0704 
Fax: 780-883-0705 



Learning Bridges 




By: 



Date 



Mid-continent Laboratory, Inc. 




By: 
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SCHEDULE A 



Background 

In an effort to raise student achievement, two public elementary schools in an urban school district in 
Minnesota have opted to enroll their teachers in a standards-based, online teacher professional 
development model created by Learning Bridges. This evaluation will follow participating teachers, and 
teachers at control sites, over a two-year period to examine the impact of this professional development 
model on teacher and student learning. 



Purpose of the Evaluation 

The overall purpose of the proposed evaluation is to assess the impact of the Learning Bridges standards- 
based professional development model on teaching and learning. 

The primary evaluation questions are: 

1 . How was the Learning Bridges professional development model implemented? 

2. To what extent is participation in the Learning Bridges online professional development model 
related to changes in teacher knowledge and skills? 

3. To what extent is participation in the Learning Bridges online professional development model 
related to higher levels of student achievement! 



Review of the Literature 



Characteristics of Quality Teacher Professional Development 

The National Staff Development Council (NSDC, 2000) has established 27 standards for effective staff 
development to support schools and districts in heeding its call to provide staff development grounded in 
research. As described by Hirsch (2000), these standards are organized into three areas: context, process, 
and content. The context standards refer to organizational characteristics that support the implementation 
and effects of professional development, such as strong leadership to obtain support and motivate staff. 
The process standards discuss how professional development should be delivered, such as opportunities 
for teachers to work collaboratively. Finally, the content standards refer to educator knowledge and skills 
that should be addressed, such as preparing teachers to use research-based teaching strategies appropriate 
to their teaching contexts. Although these standards were designed to guide schools and districts in 
identifying quality professional development, the NSDC advocates that regardless, the effectiveness of 
staff development should be judged by its influences on student achievement. 

For the past several years, researchers have sought to identify the attributes of effective professional 
development. In a national survey of 1000 teachers who participated in the federal Eisenhower program 
for professional development in math and science, Birman, Desimone, Porter, and Garet (2000) asked 
teachers to describe the effects of their staff development experiences on their knowledge and skills and 
on the changes they had made in teaching practices. Based on the results, the researchers described 



1 



effective staff development as having appropriate duration, subject-matter content, active teacher 
learning, coherence with school improvement, and collective teacher participation. The researchers noted, 
however, that only a small minority of teachers had participated in staff development that had all of these 
characteristics. In a study by Borko et. al. (1997), teachers were introduced to instructional practices in a 
workshop, and then tried out the strategies in their classrooms, followed by discussion at a later 
workshop. This was an effective approach: teachers 1 professional knowledge is developed and intertwined 
with classroom context which helps support the transfer of general principles of teaching to applications 
in the classroom (Putnam & Borko, 2000). Roskos and Bain (1998) examined other elements of 
professional development in assessing the degree to which such opportunities stimulate and support 
teachers' thinking and reflection. In their study of 16 elementary school teachers participating in 
professional development on curriculum models, the extent to which the teachers had the opportunity to 
be intellectually engaged and to think about and discuss new concepts was important to teacher learning. 
Wilson and Berne (1999) documented the positive effects of professional development involving teacher 
collaboration. Thompson and Zeuli (1999), however, pointed out that collaboration by itself does not 
necessarily improve teacher learning. Based on these and other studies, quality professional development 
has come to be associated with learning opportunities that include: active teacher engagement and 
reflection, direct applications to the classroom, teacher collaboration, sufficient duration, and alignment 
with school improvement goals. 

Despite these findings, Guskey (1997) noted that the research to date has resulted in neither clear nor 
definitive answers about how professional development can result in school improvement and lasting 
change. The reasons for this lack of clarity about the impact of professional development are: 41 1) 
confusion about the criteria of effectiveness, 2) the misguided search for main effects, and 3) the neglect 
of quality issues" (p. 34). The most common criteria for professional development effectiveness are the 
participant approval ratings of a program, changes in attitudes, and the degree to which the new skills 
were actually implemented; student learning is rarely examined (Guskey, 1997). The emphasis on the 
main effects of professional development (i.e., those that are commonly reported across programs) 
obscures important knowledge about the conditions under which staff development is most likely to 
influence student learning. Finally, most studies document only the presence or absence of desired 
features, such as teacher collaboration, and not the quality of these features, such as the intensity, 
duration, or fidelity of implementation. According to Guskey, finding the answer to what makes for 
effective professional development requires more studies that include rigorous research designs. 

Professional Development and Standards-Based Education 

The nature of professional development is also being shaped by the adoption of K-12 content standards by 
states. As described by Hawley and Valli (1999), the new higher standards require students to acquire 
more complex knowledge and problem-solving skills than in the past, and consequently, instruction needs 
to have a greater emphasis on student understanding. Teachers in standards-based school systems must be 
able to address curricular outcomes, prepare students for standards-based assessments, design lessons that 
cover complex content and result in deep student understanding, provide opportunities for discussion, 
practice effective classroom management, and assure that all students achieve (Wilson & Ball, 1996, p. 
122). These teachers, however, do not feel prepared to meet the new challenges of standards-based ' 
education. In data collected by the National Center for Education Statistics (U.S. Department of 
Education, 1998) the need for more intense professional development related to educational standards was 
clear. According to this 1998 national survey, although teachers were most likely to participate in 
professional development activities that focused on reform topics such as standards and assessment, the 
majority of these activities lasted for only a day, and only 36% of teachers felt very well prepared to 
implement state or district curriculum and performance standards. Most policy makers and education 
researchers agree that a critical factor in the success of standards-based reforms is professional 
development that addresses teachers' needs to teach in new ways that can help all children achieve to high 
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standards (NCTAF, 1996). In short, standards-based K-12 reforms require that professional development 
programs adapt to the changing needs of teachers. 

For professional development to adequately address teachers' needs, Dougherty and Young (1998) 
emphasized that the activities need to be aligned with the content standards that are already in place. 
Based on a survey of 800 teachers by the National Foundation for the Improvement of Education (NIFE), 
teacher and student growth are supported by opportunities for teacher learning that is closely connected to 
a school's goals for improvement and to teachers' more immediate needs to implement specific standards 
in their classroom (Renyi, 1996). To this end, Renyi, executive director of NIFE, urged schools to use 
student test scores to indicate where students need help and then design teacher professional development 
programs to address those needs ("State Councils," 1999). Teachers in this study concur with the 
importance of linking professional development with school improvement as assessed by student 
achievement: 73% of teachers identified helping students learn as their primary motivation for growing as 
a professional. 

The process for translating this focus on students' test scores into changes needed in instruction, however, 
is not clear. Grant (2000) summarized studies showing that while an emphasis on test scores influenced 
teachers to narrow the content they teach, it had little effect on how they were teaching. In studying the 
perceptions of teachers in New York state about the state assessments, Grant found that teachers were 
uncertain about how to use student scores to modify their teaching and uncertain about approaches that 
might improve student performance. If schools are to meet their goals of helping all students achieve high 
educational standards, then teachers need professional development that meets their needs by explicitly 
aligning with standards and providing more specific instructional strategies that target areas of low 
performance on tests of student achievement. 

Linking Standards-Based Professional Development to Teacher and Student Learning 

The new level of accountability that accompanies educational standards also can be seen in what is now 
expected from professional development. States are being urged to fund and support staff development 
for teachers only if it provides teachers the knowledge and skills to teach to higher standards and 
contributes to improvements in student learning (Sparks & Hirsch, 2000). Sykes (1999) expresses this 
trend in stating that "first and most obvious, the teacher-student learning connection should serve as a 
criterion for selection of professional and school development activity [italics in original]" (p. 161). The 
means of establishing this link between professional development and student achievement, however, 
have proven to be elusive (Guskey, 1997), particularly when it involves the traditional one-day workshops 
led by outsiders (Wilson & Berne, 1999). As Harwell, D'Amico, Stein, and Gatti (2000) report, very few 
studies of teacher professional development even attempt to measure effects on student achievement. 

One such study describes some large-scale school improvement efforts which resulted in higher student 
achievement by focusing professional development on those areas which are "proximal" to students (e.g., 
subject matter, instruction) and by organizing wide-scale teacher implementation (Joyce and Showers, 
1995, p. 63). The authors contended that school improvement based only on "distal" factors, such as 
changing standards and curriculum without teacher training, will not influence student learning. Another 
study, this one of Community School District #2 in New York City by Resnick and Harwell (1998), also 
demonstrated the positive effects on student achievement of professional development aimed at standards- 
based education. In their analysis of student achievement test results, the authors found that in the subject 
areas where professional development and accountability more clearly reflected the standards (e.g., in 
literacy), overall student performance was higher than in subject areas less emphasized by professional 
development (e.g., in mathematics). Finally, in Yoon and Resnick's (1998) study of professional 
development for middle school mathematics teachers in California, "participation in staff development 
and the use of 'reform' methods of instruction were positively associated with achievement" (p. 1 9). 
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Thus, while there are some studies that have demonstrated the link between standards-based professional 
development and student learning, it is difficult to draw conclusions as to what exactly accounts for this 
relationship. Is it the focus on standards? Or simply having a focus on areas of low performance? For 
example, in another emerging body of research, it appears that the simple fact of narrowing the focus of 
teaching to a particular subject area (e.g., language arts), or particular skills within a subject area (e.g., 
reading comprehension) will improve students' scores (Palmer, 1999). Clearly, there is much to learn 
about what types of professional development promote teacher and student learning. 

Teacher Access to Quality Professional Development 

Although research is accumulating on what features characterize effective professional development and 
what motivates teachers to participate, teachers' access to such professional development is limited. As 
indicated in a report on what states are doing to help teachers improve their instruction, most K-12 
teachers do not have access to professional development opportunities that can influence both their own 
learning and that of their students (Hirsch, Koppcich & Knapp, 1998). In many school districts, staff 
development is fragmented and delivered in brief inservice activities. For example, in a 1998 national 
survey, although 99% of teachers had participated in some form of professional development in the 
previous 12 months, two out of three teachers reported that the total duration of their involvement was 
eight hours or less, or the equivalent of a one day workshop (U.S. Department of Education, 1999). Many 
districts do not have the resources to provide for teacher release time, and instead, rely on a few 
designated inservice days. Using technology to alleviate this problem by providing online access to 
professional development anytime, anywhere, is an option being explored by educators at all levels. 
Although the current research on technology in K-12 education emphasizes teachers' use of technology in 
the classroom, there is little research on its use as a means of delivering professional development (Marx, 
Blumenfeld, Krajcik, & Salloway, 1998). Exceptions are studies of the online teacher networks that are 
becoming more prevalent (e.g., Webber & Robertson, 1998; Taylor et al., 1997). As Putnam and Borko 
(2000) pointed out, web-based technology has great potential to provide inservice teachers with access to 
the expertise of others in the field of education. At the same time, technology by itself is not a panacea for 
increasing teacher knowledge. Teachers will not be motivated to use online technology if they view it as 
merely an add-on to existing teacher learning activities without additional benefits to their instruction 
(Schlager, Fusco, & Schank, 1999). The content must be relevant and useful in their professional lives 
(Marx, et al, 1998). Based on their research, Marx, et. al. recommend that for online professional 
development to be effective it should incorporate multimedia case studies, scaffolding (e.g., posing of 
questions to promote teacher reflection), and software designs that model the innovations teachers are 
learning. The technology to provide online learning already exists and is being used widely in business 
and increasingly in higher education. As districts serving K-12 populations search for ways to increase 
teachers' access to quality professional development, they too, are turning to online solutions. 

The Professional Development Model 

The Learning Bridges professional development model includes an Instructional Intervention Plan (IIP) 
and a web-delivered professional development program. All teachers in the treatment group will receive 
the IIP and participate in the professional development component; teachers in the control group will 
receive neither. Each of these components is described below. 

Instructional Intervention Plan (IIP) 

At the beginning of the study (fall 2001), schools in the treatment group will receive an Instructional 
Intervention Plan (HP) prescribing instructional strategies aligned to target areas of low student 
performance on the district's grade level benchmarks as indicated by district and state assessments. 
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The HP is unique in that it links actual student achievement data with research on effective instructional 
practices. After examining students' tests scores at each grade level, trained consultants develop an IIP 
that identifies specific instructional strategies (e.g., concept attainment, advance organizers, etc.) that have 
been demonstrated to improve student achievement in the area of low performance (e.g., reading 
comprehension, math computation). Utilizing data about the effects of each instructional strategy on 
student achievement (i.e., effect size), the IIP provides teachers with a list of the strategies that have 
demonstrated the greatest impact in a particular area of need. The IIP also provides a bibliography on the 
relevant teaching practices, general information regarding instruction (e.g., brain research, multiple 
intelligences), and recommendations for professional development. 

The development of an IIP that links student achievement data with educational standards'and research on 
effective instruction is facilitated by a database developed by Learning Bridges (2000), a professional 
development company created by and for educators. The Learning Bridges Aligned Instructional 
Database (2000) links specific teaching practices to state standards in mathematics and language arts 
based on research on the effects of these practices on student performance (Darling, personal 
communication, July 21, 2000). These instructional practices were identified on the basis of studies that 
reported the effect sizes associated with changes in student performance. The first step in developing this 
database was to link content standards from individual states to the McREL Compendium of Standards 
(Kendall & Marzano, 1997) using another database developed by McREL for Achieve Communications 
(2000). Then, a team of nine content and instructional specialists, all with advanced degrees in education 
and an average of 25 years of experience in the field, analyzed the research and mapped the 25 most 
effective teaching practices to each of the content standards in language arts and mathematics based on 
the type of knowledge being assessed and the way in which the brain would process this information. The 
team also linked each of the teaching strategies to the instructional frameworks of Joyce, Weil, and 
Calhoun (2000) and Marzano (1998). The final result was a map of the links between effective 
instructional strategies, student achievement, and individual state standards that could be used to make 
specific recommendations for improving student performance through changes in teacher practices at 
each grade level. 

Online Professional Development Modules 

In addition to receiving the IIP, teachers from schools assigned to the treatment group also will participate 
in a two-year project (hat includes the completion of four web-delivered professional development 
modules developed by Learning Bridges to teach the specific, standards-based instructional strategies 
recommended in the plan. Each module offers a structured environment for learning a single instructional 
strategy, has context and examples related to both language arts and mathematics, and comes in 
elementary and secondary versions. The developers of the modules report that it takes about 8 to 10 hours 
to complete all of the course activities, which are scheduled over a two-week period. For the proposed 
study, teachers will be required to select four different modules from the top 25 instructional practices 
(i.e., 25 highest effect sizes) recommended by the IIP for their school and grade level. Teachers must 
begin their first module within the first four weeks of the school year and complete two modules each 
academic year. 

Although web-delivered, the modules include offline as well as online components. Online features 
include tutorials on the particular instructional strategy, pre/post-tests of teachers' declarative knowledge 
of the instructional strategy being presented, learning activities that require teachers to develop lesson 
plans targeted to the strategy being learned, and online discussions with other participating teachers and 
the course instructor. Offline, teachers are required to try out their new instructional strategies in their 
own classrooms before returning to the online course to share and reflect upon what they have learned. 
Teachers receive a Course Proficiency Score at the end of the course representing their procedural 
knowledge of the targeted instructional strategy and also complete an online evaluation of the course. The 
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University of California at Bakersfield will provide accreditation for the modules and one hour of 
graduate credit for each module, provided that teachers fulfill the course requirements and wish to pay the 
tuition. 

Evaluation Design 

This evaluation includes both formative and summative components in examining the quality, utility, and 
impact of the Learning Bridges standards-based professional development model. Formative data will be 
gathered to document the implementation process. Summative data collection will focus on the impact of 
this professional development model on teacher and student learning. This aspect of the evaluation will 
employ a quasi-experimental design (i.e., treatment and control groups) in examining the impact of 
participation in the Learning Bridges professional development program. 

Guiding Evaluation Questions 

This evaluation is guided by three key questions, one formative and two summative: 

1 . How was the Learning Bridges standards-based professional development model implemented} 
(formative) 

a. What is the context in which the professional development takes place and for teachers as 
they put what they are learning into practice? 

b. What is the nature of the treatment (e.g., quality, intensity and duration of participation)? 

c. Which features of the Learning Bridges standards-based professional development model 
have been identified as most effective in promoting the desired outcomes around 
improved teaching and learning? 

2. To what extent is participation in the Learning Bridges standards-based professional development 
model related to changes in teacher knowledge and s/a/fr?(summative) 

3. To what extent is participation in the Learning Bridges standards-based professional development 
model related to higher levels of student achievement! (summative) 

Sample 

The sample for this evaluation will include four urban elementary schools— two in the treatment and two 
in the control group— and approximately 150 teachers and 2000 students. Control schools will be selected 
on the basis of matching characteristics (e.g., percent minority, percent eligible for free and reduced price 
lunch, percent English Language Learners, and student achievement). 

Data Collection Procedures and Measures 

Summative data on teacher knowledge/skills, and instructional practices and student achievement will be 
tracked over time at the treatment and control schools including baseline assessments in the fall of 2001 
and follow-up in the spring of 2002 and 2003. Formative data on the context, nature, and effectiveness of 
the professional development model will be collected annually at the treatment sites. To reduce attrition 
of participants over the course of the two-year study, teachers at the control sites will receive honorariums 
for completing each stage of the data collection. See Table 1 for a summary of measures and data sources. 
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I Table 1: Key measures and data sources. 
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Summative Data 



Teacher-level data collection. Several data sources will be used for collecting and validating data at the 
teacher-level. The primary data sources for both the experimental and control groups include: (1) an 
Instructional Content and Strategies Report, (2) a teacher survey, and (3) sampling records. Data from the 
professional development course will be used for validation of measures. 

The two key variables for monitoring teachers* instructional practices are teachers ' use of the targeted 
instructional strategies (TIS) and content coverage. The measure of TIS use is a composite variable that 
includes both "appropriate use" of the selected instructional strategies, as assessed by the fidelity of 
implementation, and the "intensity of use" in each subject area. Content coverage is a measure of the 
breadth of material addressed during the implementation and is intended to assess the extent to which 
teachers narrow their focus on particular content areas. For both measures, baseline data will be collected 
at the beginning of the evaluation (fall 2001) with post-data being gathered at the end of each year (spring 
2002, spring 2003) via an Instructional Content and Strategies Report (ICSR) that is embedded into the 
teacher survey. The ICSR asks teachers to note which of the 25 TIS they used during the school year, how 
often, and for how long. It also asks teachers to report on the nature of TIS use to assess the fidelity with 
which these strategies were implemented in the classroom. The ICSR also asks about the breadth of 
content covered during that same period of time. 

Validation of the fidelity component of teachers ' use of the targeted instructional strategies will occur 
through comparison to the proficiency score that teachers receive as part of the professional development 
course. This overall Course Proficiency Score assesses participants' procedural knowledge and is 
assigned by the course instructor. The rubric to assess teachers' Course Proficiency Score is the basis of 
four criteria: (1) appropriate use of the TIS in the classroom, (2) quality of teachers' TIS lesson plans, (3) 
teachers' ability to be reflective as noted in their project write-ups, and (4) teachers' grasp of the TIS as 
noted by the feedback they provide to others in the online discussions. Validation of the intensity of use 
and content coverage will come via parallel measures on the self-report teacher survey. 

A self-administered teacher survey given at the beginning of the evaluation and end of each year (fall 
2001, spring 2002; fall 2002, spring 2003) will provide data on teacher demographics (gender, race, 
grade level), teacher preparedness (years of teaching this subject area, knowledge, prior use of, and 
professional development on the targeted instructional strategies, other professional development), 
context of the implementation (school climate, subject area, grade level, course structure), and conditions 
that support effective implementation of the targeted instructional strategies (teacher reflection, teacher 
collaboration, connection of the professional development process to the classroom, links to teachers' 
instructional and school improvement goals, administrator support, teacher beliefs about student learning, 
teacher efficacy related to standards, teacher ratings of subject area knowledge). The teacher survey also ' 
will include alternative measures to validate the indicators of teachers' use of TIS and content coverage 
gathered via the ICSR. Validation of teacher preparedness measures comes from a review of teachers' 
scores on the online course's Declarative Knowledge Test, which assesses their grasp of key concepts and 
vocabulary. Validation of supportive conditions comes from data collected as part of the online course 
assignments {connection to the classroom), teacher projects and course evaluation (teacher reflection), 
and records of participation in online discussions (teacher collaboration). Additional survey data to be 
collected from teachers in the treatment group includes: prior use of and proficiency in using technology, 
attitudes towards technology (preparedness)', actual time spent on each professional development module 
(dosage), and teachers' perceptions of the effectiveness of the professional development course 
(effectiveness). 

Teacher focus groups will be conducted annually in the spring to obtain feedback from participating 
teachers on the effectiveness of the Learning Bridges professional development model Teachers' level of 
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participation in the professional development model, or dosage, will also be tracked via project records 
and validated by the teacher survey. Dosage will be measured in the number of modules completed; for 
the control group the dosage will be zero, and for the treatment group up to four modules. 

Student-level data collection. The school districts involved in this evaluation maintain a database of 
student information by grade level and school. This database will be the source of data for the key 
student-level measures: student socioeconomic status (SES) (eligibility for free or reduced price lunch), 
student race (five major categories), and student achievement. 

Measures of student achievement will come from the state's two assessments; both of which are aligned 
with the state standards (see Table 2, next page). Proficiency scores from the reading and mathematics 
portions of the Minnesota Comprehensive Assessment (MCAs), administered in grades 3 and 5, and the 
Minnesota Basic Skills test, administered in grade 8, will be used to measure changes in student 
achievement over the course of the project. In addition, changes in student achievement on nationally- 
normed, standardized tests (the Northwest test in Minneapolis and the Metropolitan Achievement Test, 
Version Seven (MAT7) in Saint Paul) will also be examined. Although not aligned with the state 
standards, student performance on these tests is an important indicator of student learning. Normal Curve 
Equivalent (NCE) scores for language arts and math, and, if necessary, NCE scores for subtests (e.g., 
vocabulary, reading comprehension, math procedures, etc.) will be used for the analyses of student 
achievement on the standardized tests. NCE scores are normalized standardized scores with a fixed 
relationship to percentile ranks and equal intervals. The standardized nature of NCEs allows for 
comparisons across different tests and subject areas, and is particularly appropriate for use in multivariate 
analyses. 

Baseline data from both the state assessments and the nationally-normed, standardized achievement tests 
for the treatment and control school will be extracted from these databases in the spring of 2001 with 
follow-up data being extracted in the spring of 2002 and 2003. 

Project-Level Data. The evaluation design also includes the collection of data regarding the 
implementation of the Learning Bridges professional development model. Such data will include 
information on the context of implementation, the nature of the professional development model, and its 
effectiveness in bringing about the desired improvements in teaching and learning. Data to be analyzed in 
this regard includes school characteristics (e.g., percent minority, percent eligible for free or reduced price 
lunch), teacher quality (e.g., years of teaching, teacher certification) and school support for 
implementation, level of participation in the Learning Bridges professional development program, 
participation in other professional development, the nature of all professional development (e.g., quality, 
intensity, duration), and teacher perceptions of the program effectiveness. School-level data will be 
gathered annually from school records. Data on the nature of the Learning Bridges professional 
development program and teacher participation will be collected from annual program records. All other 
data on teacher quality, school support for implementation, and participation in other forms of 
professional development will be gathered via the teacher survey described above. Teacher perceptions of 
program effectiveness will be gathered by the annual focus groups also describe above. 
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Table 2: Sources of Student Achievemeni 


tData 




Treatment 


Control 




Minneapolis Public Schools 


Saint Paul Public Srhnnfo 


VJI dUC 

I pvpI 


Anderson Open 

(K-8) 


Bryi 

r 

\ 


i Mawr 

K.-5) 


Srhnok fTRTYi 

ij VlllfU IS 1 I mJmJ J 




students 
2001 


i esis 

ad ministprpd* 

MUlllllll JVCI cu 


is umDer 
students 
2001 


i esis 
administered* 


Tests 
administered* 


PK 


11 




29 






K 


95 




111 




• 


1 


81 




76 






2 


89 


Northwest 


75 


Northwest 


MAT7 


3 


78 


Northwest 
MCA 




Northwest 
MCA 


MAT? 
MCA 


4 


67 


Northwest 


80 


Northwest 


MAT7 


5 


59 


Northwest 
MCA 


74 


Northwest 
MCA 


MAT7 
MCA 


6 


75 


Northwest 




na 


MAT7 


7 


71 


Northwest 




na 


MAT7 


8 


72 


BST 




na 


MAT7 
BST 


PK-5, 8 
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* tests administered in the spring (March/April) 



Analytical Strategy 
Formative Evaluation 

1 . How was the Learning Bridges standards-based professional development model implemented! 

a. What is the context in which the professional development takes place and into which 
teachers put what they are learning into practice? 

b. What is the nature of the treatment (e.g., quality, intensity, duration)? 

c. Which features of the Learning Bridges standards-based professional development model 
have been identified as most effective in promoting the desired outcomes around 
improved teaching and learning? 

Data analysis for the formative evaluation will be descriptive in nature resulting in narrative descriptions 
of the context, nature of the intervention, and effective features. 
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Summative Evaluation 



2. To what extent is participation in the Learning Bridges standards-based professional development 
model related to changes in teacher knowledge and skills? 

3. To what extent is participation in the Learning Bridges standards-based professional development 
model related to higher levels of student achievement! 

Prior to conducting the primary analyses to examine the impact of participation in the Learning Bridges 
professional development model on teacher knowledge/skills and student achievement, sampling and 
context data will be used to verify the comparability of the treatment and control sites. 

The primary analyses of impact on teacher knowledge and skills will examine differences between 
treatment and control groups. This will involve a statistical comparison of the change in teacher 
knowledge and skills in treatment and control sites using analysis of covariance to control for level of 
participation {dosage), teacher preparedness and school support for the implementation. Secondary 
analyses will explore other factors that might be correlated with differences in teacher knowledge and 
skills. 

The primary analysis of impact on student performance will examine differences between treatment and 
control groups. This will involve a statistical comparison of changes in student performance over time in 
treatment and control sites using analysis of covariance to control for level of participation {dosage). 
These analyses will be conducted using the reading and mathematics portions of the MCAs for grades 3 
and 5, the BST at grade 8, and each district's standardized achievement test for the grades in which they 
are administered (typically grades 2-8). Secondary analyses will explore other factors that might be 
correlated with differences in student performance. 

Deliverables 

All evaluation briefs and annual reports will be distributed to participants at the conclusion of the project, 
so as not to influence the results. 

• Two Formative Evaluation Briefs (formative data and baseline summative) - January 1 , 200 1 ; 
January I, 2002 

• Two Annual Evaluation Reports (summative and formative data) - August 1, 2002; August 1, 2003 

• Presentation of annual evaluation results to school staff - September 2003 
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Tasks and Timelines 



Fall 2001 

• Identify key contacts at treatment schools (August) 

• Select control schools (August) 

• Design teacher survey and (August-September) 

• Revise Instructional Strategies and Content Report (August-September) 

• Meet with contacts at each school and administer baseline teacher survey and baseline ISCR to 
teachers at treatment and control schools (September-October) 

• Enter teacher survey and ISCR data (October) 

• Analyze teacher survey and ISCR (October-November) 

• Conduct focus groups with teachers in treatment group (December) 

• Transcribe focus group data (December) 

• Analyze focus group data (December) 

Winter 2002 

• Write Formative Evaluation Brief #1 detailing results of focus group and baseline data from 
teacher survey 

• Submit Formative Evaluation Brief #1 (January 1) 

• Set up process for obtaining student demographic and achievement data from schools 
Spring 2002 

• Administer follow-up teacher survey and ISCR to all teachers at treatment and control schools 
(May) 

• Enter teacher survey and ISCR data (May) 

• Conduct focus groups with teachers in treatment group (May) 

• Transcribe focus group data (May) 

• Obtain student data from schools (May) 

Summer 2002 

• Analyze teacher survey and ISCR (June) 

• Analyze focus group data (June) 

• Prepare and analyze student performance data (June-July) 

• Write Annual Evaluation Report #1 detailing formative and summative results 

• Submit Annual Evaluation Report #1 (August 1) 

Fall 2002 

• Administer baseline teacher survey and baseline ISCR to new teachers at treatment and control 
schools (September) 

• Enter teacher survey and ISCR from new teachers (September) 

• Analyze teacher survey and ISCR from teachers new to the project (October-November) 

• Conduct focus groups with teachers in treatment group (December) 

• Transcribe focus group data (December) 

• Analyze focus group data (December) 
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Winter 2003 

• Write Formative Evaluation Brief #2 detailing results of focus group and baseline data from 
teacher survey 

• Submit Formative Evaluation Brief #2 (January 1) 

• Review process for obtaining student demographic and achievement data from schools 
Spring 2003 

• Administer follow-up teacher survey and ISCR to all teachers at treatment and control schools 
(May) 

• Enter teacher survey and ISCR data (May) 

• Conduct focus groups with teachers in treatment group (May) 

• Transcribe focus group data (May) 

• Obtain student data from schools (May) 

Summer 2003 

• Analyze teacher survey and ISCR (June) 

• Analyze focus group data (June) 

• Prepare and analyze student performance data (June-July) 

• Write Annual Evaluation report detailing formative and summative results 

• Submit Annual Evaluation report (August 1) 

Fall 2003 

• Present results to treatment and control schools (September) 

Organizational Capacity and Staff 

The Mid-continent Laboratory (MCL) and the Mid-continent Research for Education and 
Learning (McREL) 

Mid-continent Laboratory (MCL) is a wholly-owned, for-profit subsidiary of McREL with full access to 
McREL resources. McREL is a nonprofit organization established to improve the quality of education 
through the application of the best available knowledge from research, development, and experience. 
Since it was founded in 1966 it has been a service-oriented, client-driven organization that focuses on 
conducting research, disseminating information, and providing technical assistance to state and local 
agencies and K-12 schools in its seven state region of Colorado, Kansas, Missouri, Nebraska, North 
Dakota, South Dakota, and Wyoming. Its mission is to work with education leaders and educators in 
designing and implementing innovative educational strategies and systems that meet the needs of our 
ever-changing society. McREL focuses its programmatic efforts around curriculum, learning and 
instruction; rural and small schools; science and mathematics; evaluation; and, applied research. 

McREL is part of a national network of educational laboratories and centers, funded by the Office of 
Educational Research and Improvement of the U.S. Department of Education, working on the cutting 
edge of educational programs and services. Research and evaluation have been central to McRELs work 
since its inception. 

McREL's core staff of 70 professionals includes nationally recognized leaders in education, research, and 
related fields. The Laboratory's main offices, located in Aurora, Colorado, maintain extensive computer 
technology, communication and production capabilities, and information resources for supporting staff 
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work. Additional information regarding McREL is available through our home page on the World Wide 
Web at http://www.mcrel.org. 

Qualifications of Key Staff 

The staff for this evaluation include: 

• Elisabeth Palmer, Principal Evaluator 

• Judy Northup, Evaluator 

• Dawn Fries, Evaluation Assistant 

Qualifications of senior staff include (see attached vita): 

Project Director, Elisabeth A. Palmer, Ph.D., is a Principal Evaluator at McREL. Dr. Palmer earned a 
Ph.D. in Sociology from the University of Minnesota. For the past 10 years she has conducted and 
disseminated educational research, policy, and program evaluations at the local, state, national and 
international level. Dr. Palmer was a researcher with the 1992 U. S. IE A Computers in Education Study 
and has conducted numerous studies of educational technology, teacher professional development, and 
student assessment. Her particular skills involve designing and managing studies involving multiple 
methods and sites, survey research, and advanced statistical analyses. Dr. Palmer will design and direct 
the study. 

Judy Northrup will manage the data collection, assist with data analysis and reporting. Dawn Fries will 
assist with data entry, analysis, and reporting. Robyn Alsop will coordinate consultant agreements with 
teachers at control sites. 

Year 1 (FY 2001-2002) Staffing days: Palmer 28, Northrup 16, Fries 16; Alsop 2 
Year 2 (FY 2002-2003) Staffing days: Palmer 37, Northrup 23, Fries 34; Alsop 4 
Year 3 (FY 2003) Staffing days: Palmer 32, Northrup 25, Fries 30; Alsop 4 
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